Objective¶
Explainability/interpretability overview and demonstration of ML tools in the Azure stack used at Avanade.
I. Background ¶
Do to our unique relationship with Microsoft, we have been given direct access to product owners for Microsoft's cutting edge machine learning, interpretability, and explainability tools including interpret-ml, interpret-community, and the azureml sdk (May Hu, Mehrnoosh Sameki, Ilya Matiach).
Why do we need Intepretable ML? ¶
"The goal of science is to gain knowledge, but many problems are solved with big datasets and black box machine learning models. The model itself becomes the source of knowledge instead of the data. Interpretability makes it possible to extract this additional knowledge captured by the model."
- Christoph Molnar, ‘Interpretable Machine Learning’
Applications
- Financial Services/Banking (fraud detection)
- Marketing (user engagement)
- Healthcare (individualized medicine and tracking)
- Epidemiology (disease outbreak modeling)
Benefits
- Validation of domain knowledge
- Provides actionable evidence
- Guides data practices and feedback
SHAP! ¶
SHapley Additive exPlanations
Gives both globally and locally accurate and consistent feature importance values derived from individual contributions (drawn from Lloyd Shapley's work in combinatorial game theory).
Ideal for use with opaque models (boosted tree, kernel-based, NN, etc).
MSFT's Interpretability Offerings ¶
Microsoft addressed the need for a unified API that makes it easy to get model explanation/feature importances based on various model types, built in to their machine learning platform.
- From the SDK
pip install --upgrade azureml-sdk[explain,interpret,notebooks]
- Only interpretability package
pip install interpret-community
Using the TabularExplainer object, the model type is detected and the appropriate SHAP explainer is selected to generate feature importances.
| Original Model | Invoked Explainer |
|---|---|
| Tree-based models | SHAP TreeExplainer |
| Deep Neural Network models | SHAP DeepExplainer |
| Linear models | SHAP LinearExplainer |
| None of the above | SHAP KernelExplainer |
This package also supports a Mimic Explainer (Global Surrogate) and a Permutation Feature Importance Explainer (PFI), both of which are model-agnostic and will be covered later.
II. In Action ¶
Dummy Example ¶
Scikit-Learn Breast Cancer Binary Classification
Pip installations:
pip install numpy, pandas, sklearn, lightgbm, interpret-community[visualization]
import numpy as np
import pandas as pd
import sklearn.datasets as datasets
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, classification_report
import lightgbm as lgbm
# Load and partition data
data = datasets.load_breast_cancer()
X = data.data
y = data.target # 0 = malignant, 1 = benign
feature_names = data.feature_names.tolist()
classes = data.target_names.tolist()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
X_train, X_valid, y_train, y_valid = train_test_split(X_train, y_train, test_size=0.25, random_state=42, stratify=y_train)
# Model training
clf = lgbm.LGBMClassifier()
clf.fit(
X=X_train,
y=y_train,
eval_set=[(X_valid, y_valid)],
eval_metric='auc',
feature_name=feature_names,
verbose=25
)
y_pred = clf.predict(X_test)
print('\nConfusion Matrix: \n', confusion_matrix(y_test, y_pred))
print('\nClassification Report: \n', classification_report(y_test, y_pred))
# Feature Importance (SHAP)
from interpret_community import TabularExplainer
explainer = TabularExplainer(clf, initialization_examples=X_train, features=feature_names, classes=classes)
# Global Feature Importances
global_explanation = explainer.explain_global(X_train)
display(pd.DataFrame.from_dict(global_explanation.get_feature_importance_dict(), orient='index', columns=['SHAP Value']).head(10))
# Local Feature Importances (for predicting benign class)
shap = global_explanation.local_importance_values[1]
df_shap = pd.DataFrame(shap, columns=feature_names)
display(df_shap.head())
# Visualization
from interpret_community.widget import ExplanationDashboard
ExplanationDashboard(global_explanation, clf, datasetX=X_train, trueY=y_train)
Avanade Interpretability VM (Demo) ¶
III. Other Approaches ¶
Different Methods ¶
Local Interpretable Model-agnostic Explanations (LIME)¶
Explainable surrogate models are trained on the predictions of the opaque model, therefore allowing local, interpretable explanations. No guarantee to be globally relevant.
Global Surrogates Models (Mimic Explainers)¶
Same as LIME, but applied to global scale. Must be an interpretable model (tree or linear) that trains on the original data with the addition of the predicted label of the opaque model.
Permutation Feature Importance (PFI)¶
Shuffles dataset, feature by feature and measures effect on performance metric. Larger changes are attributable as more important features.
Diverse Counterfactual Explanations (DiCE)¶
Uses feature perturbations to give actionable outcomes on requirements to shift between classes.
ie. If credit score was > 700, user X would likely move into the "Loan Approved" classification.
Competing offerings ¶
AWS Sagemaker Debugger just recently started utilizing the shap package microsoft has integrated into azure ml, contrasting the maturity of Microsoft's early investment in explainable AI.
Oracle's "Skater", is a python package that supports local interpretation using LIME and global interpretation using scalable bayesian rule lists and tree surrogates. The documentation is rather sparse and there doesn't seem to be any momentum to expand to other methods.
Scikit-learn's built-in feature importances provide some value for simple models, but lack the depth and versatility of msft's interpret-community.
Explain like I’m 5 (ELI5) uses LIME and PFI on opaque models and offers specialized support for text classifiers. Does not offer any visual utilities or shap.
The webapp, ml-interpret, is an online-only platform where a dataset may be uploaded and a model selected to explain outcomes of opaque models, but there is practically no customizability and the user is size-restricted.
IV. Wrap-up ¶
Questions ¶
Sources/Suggested Reading ¶
Molnar, Christoph. "Interpretable machine learning. A Guide for Making Black Box Models Explainable", 2019
Lundberg SM et al. "A Unified Approach to Interpreting Model Predictions", NIPS 2017
Shapley, LS. "Notes on the n-Person Game -- II: The Value of an n-Person Game", 1951
Packages ¶
https://github.com/slundberg/shap
https://github.com/interpretml/interpret-community
https://github.com/interpretml/DiCE
https://github.com/TeamHG-Memex/eli5
https://github.com/oracle/Skater
http://ml-interpret.herokuapp.com/